Latent Dirichlet Reallocation for Term Swapping

This paper is an extended abstract of a work in progress, which proposes latent Dirichlet reallocation (LDR), a probabilistic model for text data from different dialects over a shared vocabulary. LDR first uses a topic model to allocate word probabilities to vocabulary terms; it then uses a subtopic model to allow for a possible reallocation of probability between a few potentially swappable terms between dialects. An MCMC inference procedure is derived, combining Gibbs sampling with Hamiltonian Monte-Carlo. Finally, we demonstrate the ability of LDR to correctly switch the probabilities for swappable terms under the subtopics using a toy example. International Workshop on Statistical Machine Learning for Speech Processing (IWSML) This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c ©Mitsubishi Electric Research Laboratories, Inc., 2012 201 Broadway, Cambridge, Massachusetts 02139